1. Technical Field
The present invention relates generally to enterprise data management.
2. Background of the Related Art
A critical information technology (IT) problem is how to cost-effectively deliver network wide data protection and rapid data recovery. In 2002, for example, companies spent an estimated $50 B worldwide managing data backup/restore and an estimated $30 B in system downtime costs. The “code red” virus alone cost an estimated $2.8 B in downtime, data loss, and recovery. The reason for these staggering costs is simple—traditional schedule based tape and in-storage data protection and recovery approaches can no longer keep pace with rapid data growth, geographically distributed operations, and the real time requirements of 24×7×365 enterprise data centers.
Traditionally, system managers have used tape backup devices on a periodic basis to store application and system data, or to generate volume snapshots of a primary storage. Conventional tape backup tools typically extract data from an application, dump the data into one or more tapes, and then make a catalog that associates the data and the particular tapes, as well as offset data within a tape. The application data can be re-introduced to the application at a later time in the event of a failure event. Volume snapshot tools, on the other hand, provide the ability to acquire a “snapshot” of the contents of an entire hard disk at a particular time and then store this for later use, e.g., reintroduction onto the disk (or onto a new disk) should the computer fail. The problems with these backup approaches are well known and appreciated. First, changes in data occurring after a backup or snapshot are always at risk and may be lost. Second, tape backup requires that the host application be shutdown or to be put into a backup mode for a long time period until the complete data set is copied to tape. Third, as the data size grows, the bandwidth required to offload data repeatedly, and the attendant storage required to store the complete snapshots, can become impractical quickly. Further, during a “hot” snapshot critical data can change, which may result in an incomplete update being captured (e.g., only one portion of a transaction) such that, when reintroduced, the data is not fully consistent. Most importantly, storage based snapshot does not capture fine grain application data and, therefore, it cannot recover fine grain application data objects without reintroducing (i.e. recovering) the entire backup volume to a new application computer server to extract the fine grain data object.
Data recovery on a conventional data protection system is a tedious and time consuming operation. It involves first shutting down a host server, and then selecting a version of the data history. That selected version of the data history must then be copied back to the host server, and then the host server must be re-started. All of these steps are manually driven. After a period of time, the conventional data protection system must then perform a backup on the changed data. As these separate and distinct processes and systems are carried out, there are significant periods of application downtime. Stated another way, with the current state of the art, the processes of initial data upload, scheduled or continuous backup, data resynchronization, and data recovery, are separate and distinct, include many manual steps, and involve different and uncoordinated systems, processes and operations. The volume snapshot tools are not able to recover granular objects, the tape backup tools are not capable of searching for protected objects over time. None of these tools are capable of indexing objects across multiple arrays and hosts, and they are not capable of locating protected objects that existed in any point-in-time in the past.
The current invention not only protect data in real-time in a distributed network, it is capable of capturing consistency events, and then index the protected information with time and events. By doing so, the current invention allows one to search for an object of any granularity (a database, a file, a message, a volume, etc.) across a distributed network and across history.